Cameras are commonly used to capture an image of a scene that includes one or more objects. Unfortunately, some of the images are blurred. For example, movement of the camera and/or movement of the objects in the scene during the exposure time of the camera can cause the image to be blurred. Further, if the camera is not properly focused when the image is captured, the image can be blurred.
When blur is sufficiently spatially uniform, a blurred captured image can be modeled as the convolution of a latent sharp image with some point spread function (“PSF”) plus noise,B=K*L+N.  Equation (1)
In Equation 1 and elsewhere in this document, (i) “B” represents a blurry image, (ii) “L” represents a latent sharp image, (iii) “K” represents a PSF kernel, and (iv) “N” represents noise (including quantization errors, compression artifacts, etc.).
A non-blind deconvolution problem seeks to recover the latent sharp image L only when the PSF kernel K is known. Alternatively, a blind deconvolution problem seeks to recover both the PSF kernel K and the latent sharp image L. Both the blind deconvolution problem and non-blind deconvolution problem are difficult to accurately solve because they are ill conditioned, and some information has been irretrievably lost due to zero and near zero values in the corresponding optical transfer function (“OTF”), Fourier transform of the PSF.
Further, the blind deconvolution problem is extremely difficult to solve because there are more unknowns than known variables. Further, infinitely many arrays exist that satisfy Equation (1), but only some of them make sense and are useful as solutions to the blind deconvolution problem, which seeks a PSF kernel K that characterizes blur present in the given blurry image.
Moreover, many blurry images include areas that further complicate the problem of recovering a latent sharp image L. For example, extremely bright areas where the sensor pixels reach their saturation point in the blurry image B can adversely influence the determined PSF kernel K and the resulting latent sharp image L.
Further, certain depth layers of a blurry image B will have a different blur PSF kernel. For example, consider a blurry image B that includes something in the foreground (e.g. people) and something in the background (e.g. mountains). If the main subject in the foreground is not properly focused, both the foreground and the background of the blurry image B can be blurred, but to a different degree. Considering the previous example, it may only be desired to deblur the people in the foreground, while allowing the mountains in the background to remain blurred. Additionally, many current deconvolution methods often result in a latent sharp image L having ringing artifacts around strong edges in the image and around image boundaries.
One common approach to solving a deconvolution problem includes reformulating it as an optimization problem in which suitable cost functions are minimized in order to find a solution to the deconvolution problem. For example, the blind deconvolution problem is often solved in iterations by minimizing two cost functions in alternating fashion. More specifically, the blind deconvolution problem is often solved in iterations by minimizing a latent sharp image estimation cost function, and by minimizing a PSF estimation cost function in alternating fashion. In this example, the step of minimizing the latent sharp image cost function is equivalent to solving a non-blind deconvolution problem. Further, the step of minimizing the PSF estimation cost function is similar, except the roles of the variables (K and L) are swapped.
In addition to alternating minimization of the latent sharp image estimation cost function, and the PSF estimation cost function, it is also common to use a multi-resolution approach, where first the coarse versions of the latent sharp image L and the PSF kernel K are estimated using a heavily downsampled blurry image and then these estimates are refined at higher and higher resolution, until the full resolution of the given blurred image is reached.
A relatively common type of a cost function used for deconvolution is a regularized least squares cost function. Typically, a regularized least squares cost function consists of (i) one or more fidelity terms, which make the minimum conform to equation (1) modeling of the blurring process, and (ii) one or more regularization terms, which make the solution more stable and help to enforce prior information about the solution, such as sparseness.
Unfortunately, existing regularized least squares cost functions are not entirely satisfactory.